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[57] ABSTRACT 

The systems, methods and apparatus of preferred embodi- 
ments of the present invention provide an integrated instruc- 
tional system directed to learning a specific task or concept. 
Some embodiments are particularly suited for language 
instruction and some embodiments will accommodate a 
group of students with differing native languages. This is 
accomplished in the exemplary embodiment through the use 
of a template and database system wherein computer activity 
templates are programmed to perform a task or carry out an 
exercise. An activity template accesses data stored in a 
multilingual language database according to a student's 
needs or preferences. Embodiments of the present invention 
also provide a review method and system which optimize 
study efficiency by managing the content of review sessions 
according to each student's individual familiarity with those 
concepts, A systematic spaced review method gauges a 
student's long-term retention, imderstanding and familiarity 
with a concept by measuring, recording and monitoring the 
student's speed and accuracy of response to a prompt. Some 
embodiments also provide a concept tagging method and 
system whereby a database of words, phrases, sentences and 
other similar language constructs is analyzed to identify 
specific grammar, syntax, vocabulary or other language 
structure or concepts. Database elements are tagged accord- 
ing to these constructs for sorting and filtering according to 
a students needs. The systems and methods of the present 
invention will also relieve the instructor of the responsibiU- 
ties of monitoring student progress, tailoring materials to 
individual students and their varying levels of progress, 
developing testing materials to gauge progress and 
proficiency, and developing study aids to help students 
master particular challenges. The system and method also 
benefits students generally by providing systems and meth- 
ods which allow students to study at their own pace on an 
individual basis thereby providing an optimum level of 
challenge for all students. 

42 Claims;, 15 Drawing Sheets 
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TECHNOLOGY ASSISTED LEARNING 
BACKGROUND OF THE INVENTION 

1, The Field of the Invention 

This invention relates generally to the use of an 
educational, training, teaching or learning method and 
system, the elements of which may be implemented by way 
of computer or other conventional apparatus for use by 
individual or multiple student groups. The method and 
system may be used for instruction in any number of 
subjects, but is particularly suited to language training. The 
system employs multimedia computer technology as well as 
more conventional teaching and study aids. 

2. The Relevant Technology 

Instructional and teaching systems have been in existence 
for centuries, but their development has increased markedly 
during the "computer age" with the advent of multimedia 
technology and its many benefits. The overcrowding of 
schools, the increased knowledge expectation from students, 
and more hurried schedules increase the demand for an 
efiBcicnt teaching or learning system. In this environment, an 
efiScient learning system, such as the present invention, 
permits an individual student to learn at her own pace, rather 
than that of the group, providing an instructor with the 
opportunity to spend more time working on students' par- 
ticular needs, manage larger classes, or work with multiple 
classrooms thereby increasing efficiency and productivity. 

In the current technology, computers have been imple- 
mented in the learning process in many ways. Systems 
which present a series of static lessons separated by a 
prompt-response testing procedure which determines 
whether the student will be allowed to progress to the next 
lesson or return to additional instruction on the tested subject 
in another formal are known. These methods monitor stu- 
dent progress and disseminate additional information as the 
student progresses. 

Also known are learning systems with material indexed 
by type, item and importance factor which selects an appro- 
priate lesson according to user input and edits out parts of 
the lesson which are considered to be below the student's 
comprehension level. 

Other learning systems employ computer technology, but 
are limited in scope to particular fields of instruction, such 
as instruction in the use of computer programs or limited in 
format to specific media, such as text and simulation exer- 
cises. StiU other methods use lessons or objectives arranged 
in a predefined hierarchy. Other prior art references focus on 
monitoring and evaluation of the student rather than on 
instruction. 

The majority of learning systems in the prior art utilize a 
static lesson format which is typically arranged in a pre- 
defined order. This format ignores the individual needs of 
each student and forces the student to conform to a static 
method which may not fit her specific needs. What is needed 
is a lesson format which dynamically adapts to the student's 
strengths and weaknesses tising a system which gives the 
student more time to study the particular items which 
challenge her optimally and by providing a system which is 
not restricted by static lesson blocks. 

The prior art also lacks a system or method for ensuring 
that learned information is stored in the student's mid or 
long term memory What is needed are systems and methods 
for review which ensure that the information is retained 
beyond short term memory so the student is more likely to 
retain the information for long periods of time. 
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Prior art references also lack the ability to allow the 
student to choose from optional study programs while 
providing a method for ensuring that each item is studied for 
a proper amount of time in an efficient sequence. What is 
S needed are systems and methods which permit the student to 
specify activity sequencing. 

While it is known to edit portions of lessons which the 
student has already mastered, no existing system ensures 
that the student is consistently and properly challenged using 
10 methods to filler learning content based on changing levels 
of complexity. 

Also unknown in the prior art but needed is a method 
which aUows the instructor to efficiently prepare a database 
so that it can be tagged and filtered for use according to 
specific conceptual content. 

Also needed are systems and methods which allows the 
system to be used to simultaneously teach multiple lan- 
guages to students with differing native languages. Such a 
system would allow one system to be used universally in 
many countries to learn multiple languages or other disci- 
plines and allows the same system to be used to teach 
languages in both directions (i.e., Japanese to English and 
English to Japanese) simply by selecting the proper native 
and target languages. When configured with the proper 
databases such a system could conceivably be used to teach 
any language to any student. 

Existing systems also lack effective memorization meth- 
ods. What is needed are additional systems and methods 
which enhance memorization activities through hint and 
prompt techniques. 

SUMMARY AND OBJECTS OF THE 
INVENTION 

35 The objects and advantages of the present invention are to 
provide an integrated instructional system containing print 
materials, practice activities, and computer activities all 
linked and diirected to learning a specific task or concept. 
The combined focus of the different media and activities 

40 provides a student with multiple means of learning and 
reinforcing concepts. Each of these optional means can be 
selected to accommodate the personal learning preferences 
and needs of a particular student. 
An object of the invention is to provide a learning system 

45 that can accommodate students with differing native lan- 
guages. This is accomphshed in the exemplary embodiment 
through the use of a template and database system wherein 
computer activity templates are programmed to perform a 
task or carry out an exercise. An activity template accesses 

50 data stored in a multilingual language database. The user's 
primary language and, if the system is being used to learn a 
foreign language, the target language are selected. The 
template accesses the appropriate language databases 
according to this selection and the user is thereby able to 

55 access the system in the appropriate language. Once a 
selection has been made the student may access program 
modules in that language which control use of the activity 
templates. This feature makes the system particularly suit- 
able for multi-national classes where students in the same 

(SO class can access learning materials in their different native 
languages. 

Another object of the invention is to provide a review 
method and system which optimize study eflSciency by 
managing the content of review sessions according to each 
65 student's individual familiarity with those concepts. A sys- 
tematic spaced review method gauges a student's long-term 
retention, understanding and familiarity with a concept by 
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measuring, recording and monitoring the student's speed hints leading the student to the coaect response, if necessary. 

and accuracy of response to a prompt. If a student has As the learner answers correctly, the hints are gradually 

mastered a concept her response will be quick and accurate removed, whereas if the learner continues to struggle, more 

and the system wUl recognize this famiUarity and remove hints are added. In this way, the system is dynamically 

that concept from study until the next major review session, s customizing the learning experience to each user. 

If the student responds inaccurately and slowly the system , , . . r l • 

• *L jr jj-.' I.J ... . addition to the above objects of the invention, the 

will recognize the need for additional study on that concept . j iu a t 4u *• -n i- *l 

„ . „ -11 f , - . J ■ * -i •* • system and method of the present invention will relieve the 

and will keep the concept in active study sessions imtu it IS c*u u-r*- r j _r c 

, J . J * ■ • * J i u 11 • mstructor of the responsibihties of and perform the functions 

mastered. Inis method optimizes study time by allowing the r „ •* • * j * . -i • . • i . ■ j- 

oH.^^«* «««^^«f. #^ »u . u 11 • * °i monitoring student progress, tailoring materials o mdi- 

student to concentrate on the most challengine concepts m . • j„«i j * j *u • • i i r j i 

. •! , , „ . * 1 J . ji_ Jl vidual students and their varying levels of progress, devel- 

while the less challenging concepts already mastered by the • , . - i * j c - 

t A i J P * J * i • r * • oping testing matenals to gauge progress and proficiency, 

student are removed from study unUl infrequent review |evelopi,g study aids to help students master partieuk 

sessions. In h« manner concepts are ranked by famiharity ehaUenges He sysiem and metood ako benefits students 

in a series 01 review pools m which concepts are Stored and ^ , ... ^ j .i_ j l- i « 

u **j ' . c generally by providing systems and methods which allow 

then used in subsequent study exercises at a frequency ic ,i * tu - • ^- ^ i u • 

, . , . . 1 ^ i_ * J *j ^ r students to study at their own pace on an mdividual basis 

which IS inversely proportional to each student s mastery of u j- *■ i i r l n r n 

.'ru- • thereby providing an optimum level of challenge for all 

the concept. This process is accomplished by the system students » 

without the active knowledge or conscious effort of the 

student or instructor which further frees both to focus on an BRIEF DESCRIPTION OF THE DRAWINGS 

individualized learning experience. 20 

Another object of the invention is to provide a concept order that the manner in which the above-recited and 

tagging method and sysiem whereby a database of words, other advantages and objects of the invention are obtained, 

phrases, sentences and other similar language constructs is a more particular description of the invention briefly 

analyzed to identify specific grammar, syntax, vocabulary or depicted above will be rendered by reference to a specific 

other language structure or concepts. The constructs con- 25 embodiment thereof which is illustrated in the appended 

taining the specific language structure or concept that is drawings. With the understanding that these drawings depict 

being analyzed are, then, tagged to identify the presence of only a typical embodiment of the invention and are not 

the concept of structure so that the database can be manipu- therefore to be considered to be limiting of its scope, the 

lated and accessed according to the tagged grammar, syntax, invention will be described and explained with additional 

vocabulary or other language structure or concept. This 30 specificity and detail through the use of the accompanying 

tagging allows the system to arrange material in order of drawings in which: 

complexity and to expose the student to concepts and FIG. 1 is a typical hardware system diagram for the 

concept examples of gradually increasing complexity to present invention. 

optimize learning with a steady flow of challenging concepts piG. 2 is a graphical representation of the linked rela- 

and to avoid the distractions of material divided into sepa- 35 tionship between database tables. 

rate and distinct lessons. ^ir^ 1 u * c.l l t.- l.l 

„„., , . , HO. 3 shows the steps or the process by which the present 

Whde leammg a new discipline or language, a student invention tags content examples using the concept lagging 

becomes more proficient earlier in the learning process when system ^ 

the student learns to speak phrases or use new terminology r-x^ ' a - a 1 • . • . . , , . 

in the foreign language or target discipline from the onset of 40 ^ f diagram showmg the steps mvolved m 

instruction. An object of the present invention is to provide S^^^^f ^^"g ^^"^^"t ^^^thm the concept tagging system^ 

a method or system whereby the student can learn to speak ^^'^^3 ™' ^ ''^ P^rfo^^^^d withm blocks 320 and 330 

or use new phrases through an interactive computerized ^ ' ' 

tutor that adapts to the student's ability by offering more ^ ^ ^ ^^"^ diagram showing the steps involved in 

difficult exercises as the simpler exercises are successfully 45 refining element lists within the concept tagging system. The 

completed. This is not performed with static pre -formulated ^^^P^ i° ^I*^- ^ ^^e performed within blocks 340 and 350 of 

lessons, but in phrase-specific exercises that dynamically ^• 

adapt with each student response. For example, a phrase in FIG. 6 depicts two examples of the activity tracks which 

a foreign language or a new medical or technical term might are assigned to learning items. 

be introduced in two parts and the student asked to assemble 50 FIG. 7 demonstrates the process used to determine if an 

the phrase in the proper order. If the student successfully item progresses from the working group or is held within the 

assembles the phrase, the next time it is presented the phrase working group for further review. 

will be presented differently, perhaps divided mto several pjq g demonstrates the process used to calculate the 

more parts or contain a higher number of distractors, ^ response time of the student during exer- 

distracting false answer options, to provide a higher degree S5 

of difficulty at each iteration of the phrase. If the student ^'^ . ^ ^ - 1 . 

answers incorrecdy, the phrase is also presented differently, ^ ^^emonstrates the real time response evaluation 

typically in fewer segments with fewer distractors until the "^'^ *f P^^f determine the extent 

correct order is selected by the student. This same concept of advancement of anitem as it progresses from the working 

of adapting the activity's difficulty to gradually reach mas- 60 ^^^^P* 

tery is used further in another part of the module or system 10 is a table which demonstrates the progress of 

where a definiUon is given in a known language as a prompt learning items during a typical review session as the smdent 

for the student to input the corresponding foreign word or makes various responses. 

phrase or technical term. If the response is incorrect, the FIG. 11 is a graph showing the independent progression 

correct answer is shown to the student, after which the 65 of items through pools. 

student is prompted with a version of the correct response FIG. 12 is a flow chart showing the activity eligibility 

with portions blanked out. The student is also provided with analysis used by the present invention for choosing the next 
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activity to be presented to a student during review exercises diode display (LED), or other standard display device 

using student-specific activity sequendag. known in the industry; a text output device such as a printer 

FIG. 13 shows the steps of the student-specific concept an audio output device 120 such as a sound card and 

filtering process used by the present invention to detenmine speakers capable of emulating spoken language; data storage 
which content examples of the learning item will be used for 5 and retrieval devices 130, either direct or networked such as 

a current activity. • Jiard drives, floppy drives, tape drives and other storage 

HG. 14 is a flow diagram which discloses the logic used ^^^^^^^^^ ^ ^^f^^^^^ processing unit (CPU) 140 for executing 

by the present invention to divide phrases into iniUal seg- prograna instructions and sending and receiving instruc- 

ments which can be assembled into independently mean- ^fJ'^"^ penpheral devices; a random access 
ingfiil parts. '^^ memory 150 for ready storage and access of programs, 

nrr^ ie- n j- j . i • Operating system instructions and data; a pointing device 

FIG. 15 IS a flow diagram demonstrating the logic used to ^^^^ ^ ^ ^^^^^^^^^ ^^^^^ ^^^^^^ ^^^^^ ^^^^^ 

de^ermme how difficult to make the next question for the ^^^^^^^^ ^^^^^^^ ^^^^^ ^.^^^^^^^ ^^^^ ^^pl^^ 

ordenng parts acuvity. ^^^j^^. ^ -^^^^ ^^^j^^ ^^^^ ^ keyboard for input 

DETAILED DESCRIED ON OF THE °^ responses and selection of optional choices presented by 

PREFERRED EMBODIMENT program; a voice input device 180 such as a microphone 

for recording and digitizing the user's voice. 

Including, by express reference, the figures listed above as 

part of this detailed description. Program Structure 

20 

System Overview Important elements of the present invention include the 

use of databases and templates. The content for each subj ect 

The present invention is referred to as the technology matter to be learned, such as a foreign language, is stared in 
assisted learning (TAL) system. The TAL system may a database. This allows vast amounts of material to be 
include classroom activities, print materials, practice universaUy and easily stored, input, accessed, transported, 
activities, computer activities and simulation activities that replaced and updated. Content from the database can then be 
are linked together to provide the student with the experi- used to generate learning activities and print materials for 
ences and skills necessary to learn and understand new the target discipline. Software templates, computer pro- 
information, concepts or skflls and retain them in long-term grams which access and manipulate database information, 
memory. The system may also be used to leant to perform are used to create interactive learning activities. These 
various tasks in a target subject matter, be it a language, templates are programmed to be flexible so that they can be 
science, technology or trade. The TAL system may be configured into any number of activities using different 
implemented through a wholly computerized system, on a database content. In the preferred embodiment, templates 
single computer or networked with any number of are grouped into programs or modules which are divided 
computers, when the more conventional classroom exercises according to their learning area or purpose, 
are not convenient or possible. g^ch template, within a program module, can caU on any 

When classroom exercises are available, students are subject matter database (i.e., language database, medical 
introduced to a learning task in the classroom that they will terminology database, etc.) to draw content for presentation 
need to perform in the target subject matter, for example to the user. The templates include apphcations for: task- 
speaking simple phrases that arc required to perform a oriented learning activities, grammar exercises, vocabulary 
specific task. The teachers may use a videodisc or videotape exercises, phrase memorization and comprehension, listen- 
player or a computer to show segments of video in which ing comprehension, and creating customized learning for 
others who have mastered the task are seen performing the individualized needs, preferences, or review. The activity 
task. Audio recordings may also be used where video templates, one or more subject matter databases, individual 
capabihty is not convenient. Later the students may study leamcr progress tracking databases and the programs that 
the task in the computer lab with the TAL software described link and manipulate them comprise the TAL software, 
herein. They can record their voice as they attempt a task, xhe database contains all the content information or 
study tasks drill themselves on phrases or vocabulary, study ^^^^^^ ^^^^^ ^^^^^-^^ ^^^^^^^ ^^^^s. To maximize 
additional principles, work on their listening the number of ways the content can be used, the content is 
comprehension, and create their own study materials and organized into tables that are linked to one another in a 
audio recordings. unique fashion. As shown in FIG. 2, the prominent tables are 
. When a sufficient number of teachers are available, a the Tasks, Sentences, Words, Phrases, and Grammar Prin- 
student may practice performing the task with the teacher. ciples tables. Each of these tables holds information about 
After the simulation, the teacher may give feedback. It each instance of that type of content. For example, the 
should be noted, however, that the TAL system may be 55 Grammar Principles table contains a record for each gram- 
successfully and efBciently implemented without the aid of mar principle, which includes fields such as the name of the 
professional instructors once the system has been configured principle, the difficulty, the type, and the activity track used 
for the type of instruction desired. to teach the principle. 

Com uter Hardware two-way arrows in FIG. 2 symboUze the relationships 

60 between the tables. Typically these relationships are defined 

The computer implemented portion of the present inven- in the database with additional tables. For example, there is 

tion may be implemented on a computer hardware system as a Grammar-Sentence table that relates grammar principles to 

shown in FIG. 1. A typical computer hardware system used sentences. This table is created as a result of the grammar 

for the present invention may use any one or more of tagging process described in this description of the preferred 
numerous operating systems and microprocessors, however 65 embodiment. The Grammar-Sentence table defines a many- 

the system will typically be comprised of: a visual display to-many relationship between the Grammar Principles and 

device 110 such as a cathode ray tube (CRT), Hght emitting Sentences tables. Therefore, each record in the Grammar- 
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Sentence table refers to a record in the Sentences table as concept tagging process involves the content expert who 

well as to a record in the Grammar Principles table. This manually eliminates elements from the element lists gener- 

table also has fields that specify which words in the sentence ated in the previous steps 340. This process is shown in more 

relate to the grammar principle, and whether it is a good detail in FIG. 5. The process begins with a display of the 

example, a poor example, or a non-example. These special- 5 clement list corresponding to the first part of the query 505. 

ized links and tables enable the activity templates to draw While inspecting the list, the content expert manually rejects 

upon the database content so as to greatly increase the elements that do not correspond to the desired concept 510. 

variety of potential activities. This selection is recorded 515, and used to filter out ele- 

Use of the content from a database is maximized by '"^^^^^ ^^^^^ part-specific element Usts that are no 

drawing on it from various templates. For example, a single lo longer valid due to the rejecUons 540. This process contin- 

language sentence from the database may be used as part of for each element list 535 untU the last list has been 

a task dialogue, as a grammar translation activity sentence, ^corded 520. The mvention then finds content examples 

as a phrase sample sentence, as a Ustening comprehension ^^^^ elements of a given query 525 and displays 

distractor, as a vocabulary fiU-in-the-blank sentence, and examples 530. At this point the content expert is given 

edited to create individualized exercises. Similarly, maxi- 15 another opportunity to reject examples that do not accurately 

mum efl&ciency is achieved when a single template accesses represent the desired concept 545. The content examples that 

as many subject matter databases as possible. rejected are then recorded as examples of the desired 

concept 550 for potential use in the various activities of the 

Concept Tagging System program. 

. 20 In the preferred embodiment, the tagging program 

One important and novel aspect of the present invention responds to the search criteria by generating a list of words 

IS the ability to tag a database to identify the presence of each part of the search criteria, and allows the content 

concepmal pnnciples m the database elements. This prin- expert to refine each word list by rejecting words or word 

ciple can be applied to databases contammg elements of ^^^^^^ that do not relate to the learning concept at hand, 

many genres (i.e., mathematical equations, medical x^en all the sentences are displayed which contain words 

termmology, language grammar, vocabulary, etc.) which are ftom each of the refined word lists, for each query, in the 

referred to as learning items. In the preferred embodiment of ^.d^, specified. Finally, the content expert browses through 

the present invention textual elements are tagged for various this Ust of sentences and rejects any that do not apply. This 

grammar principles such as verb conjugation, subject process is repeated for each learning concept such as gram- 

pronouns, or prepositions for use in foreign language train- 3^ j^ar. 

Several aspects of the tagging system make it especially 
An example concept tagging system is outHned in FIG. 3. powerful and efScient: word-based rejection, automatic con- 
Prior to tagging, all the sentences (and the words they texmal filtering, and special criteria features, 
contain) are entered into the content database. When used Word-Based Rejection 

for foreign language instruction, these sentences and words 35 Word-based rejection speeds up the tagging process 

are translated and hnked to their foreign language counter- because it allows categorical rejection of sentences that are 

parts. Once this data is entered into the database, the not examples of the learning concept. For example, instead 

grammar tagging process begins. Based on his/her under- of having to pour through and reject 100 sentences out of 

standing of the grammar patterns of the language, the 300, the content expert might be able to categorically 

content expert supplies search criteria to the grammar tag- 40 eliminate the 100 sentences by simply rejecting one word 

ging program 310 that will likely identify example sentences from a word list. This is possible if all the sentences to reject 

of a particular grammar principle. have a particular word in common, which is quite often the 

Search criteria may be entered in a number of ways, but case. Based on the inventor's experience with tagging the 

in the preferred embodiment the criteria are divided into grammar of several languages during experimental use, 

individual queries of particular principles and further 45 word-based rejection consistently speeds up the tagging 

divided into query parts. Parts are space delimited while process. Moreover, the word rejection system is enhanced by 

queries are comma delimited. A typical search criteria might allowing the content expert to accept or reject sentences with 

look like the following: *te *ia, *ta og; where the criteria a particular usage of a word that has multiple meanings 

contains two queries each with two parts (the *'s act as wild (glosses). For example, in Spanish the word "como" means 

card characters similar to other common applications). Once 50 both "1 eat" and "as, like". Previously the content expert 

the search criteria have been identified and entered, the specified which gloss was intended for every sentence with 

present invention will generate element lists that match each the word "como". This content work had to be done earlier 

part of the search criteria 320. This process is demonstrated for another purpose. The grammar lagging system shows 

in FIG. 4. The invention begins by taking the first query in two instances of the word "como" in the word lists, along 

the criteria 410. This query is parsed into its parts 430 and 55 with the associated gloss. Then the content expert can 

an element list is generated 440 and stored 450 for each part choose to reject sentences that use a particular gloss of the 

of the query. After the last part in the query has been parsed word. If the grammar principle is Present -ER Verb 

464 and its element list generated and stored 460, the present Conjugation, the content expert simply selects the "I eat" 

invention will then delete elements from these part-specific instances of "como" and rejects the "as, like" instances in a 

clement lists for which a single content example docs not 60 single step rather than going through each example indi- 

contain all parts of the query 470 and 330. For example, if vidually. Time is saved by rejecting several sentences auto- 

the element is found solely in a content example sentence matically through a single gloss rejection, 

that contains grammar corresponding to only one part of the Automatic Contextual Filtering 

query and the query has multiple parts, that element will be Automatic contextual filtering expedites grammar lagging 

eliminated from the element list. This process is repeated for 65 by reducing the number of selection/rejection choices the 

each query 480, then the element lists are displayed 490 to content expert makes. Merely matching the search criteria is 

the user. Referring again to FIG. 3, the next step of the not a sufficient condition for a given element to appear in an 
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element list. The element in question must also appear in at 
least one content example, in the context of the other 
elements that satisfy the other criteria parts. All other 
candidate elements are filtered out initially before displaying 
the clement lists to the content expert. Furthermore, the 
element must appear in the context of elements that are not 
subsequently rejected by the content expert. This is ensured 
through a second filter that is reapplied with every word 
rejection. Only the elements that pass both of these filters are 
presented for consideration. Note that rejecting a single 
element in one element list could ca;ise several elements in 
other element lists to be removed from consideration. 
Special Criteria Features 

Special criteria features provide the content expert with 
powerful ways of specifying which elements to consider. 
The four special criteria features in the preferred embodi- 
ment are target word search text, definition search text, 
parent search text and verb search text. 

Target Word Search Text. This is the most basic criteria. 
To illustrate, imagine that the Spanish content expert needs 
to find all instances of the conditional tense. One target word 
search text might be "*ia*", since every conditional verb 
conjunction includes the part "la". Note that the asterisks 
here act as wild -card characters, and indicate that "la" may 
be followed and preceded by zero or more characters. 

Definition Search Text. Tliis feature searches the defini- 
tion field of each database element for specific text. For 
example, to search for all words that are examples of the 
imperative tense, the content expert might search the defi- 
nition field using the criteria "*command*" . This criteria 
would be sufiBcient, assuming that the specified definition of 
every command form included the text "command". This 
would be the case, if the content expert has previously 
defined words in the imperative tense with that term, for 
example: "tenga" defined as "have (command)", and 
"coman" defined as "eat (command)*', and so on. 

Parent Search Text. This criteria feature allows the con- 
tent expert to find all words that have the same kind of parent 
(root word). For instance, to find all conjugations of the verb 
"ser", the user would search the parent field using the criteria 
"PARENT{ser}". Any of these criteria features can be used 40 
in combination, making the search abiUties even greater. For 
example, to find all the conditional conjugations of -AR 
verbs in Spanish, the criteria would be "* 
ia*.PARENT{*ar}". Note that a period separates each ele- 
ment of the criteria. To find all subjunctive conjugations of 45 
-ER verbs, the criteria would be 
"EN{*subj*}.PARENT{*6r}". 

Verb Search Text. Verb search text allows the content 
expert to find all conjugations of a certain tense, person, or 
other characteristics. To expedite the tagging of sentences 
containing verbs, special criteria are used to identify certain 
features. Verbs are stored in a portion of the database called 
the verb table. Every verb in the verb table is conjugated 
either automatically using identified verb classes or manu- 
ally for special irregular verbs or when automatic conjuga- 
tion is not available. The conjugated forms of verbs arc then 
linked to the word table through a verb criteria table which 
identifies the conjugation type, tense, person, and verb 
parentage. While tagging, the content expert can specify 
certain criteria for a desired list of verbs and the tagging 
system will locate those verbs using the linked verb criteria 
table. For example, the content expert may specify criteria 
defining a list of words that are verbs which have regular 
conjugation, present tense and first person singular form. 
Multiple Set Search Criteria 

Another aspect of the concept tagging system is also 
important. The content expert can specify multiple sets of 
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search criteria for a single principle. Each set is separated by 
a comma. To find sentences (content examples) that illus- 
trate Spanish regular number/gender, the sets of criteria 
might be "*o *o, *a *a, *os *os, *as *as". 
Element List Rejection Optimization 

Another useful aspect of the concept tagging system is 
how the element fists are presented to the content expert. The 
shortest clement lists arc presented for consideration first. 
(Except that element lists that could only contain one 
element are not shown at all. The search criteria "hay" can 
only match a single word in Spanish, for instance.) Present- 
ing the shortest lists first minimizes the number of rejections 
the content expert makes because elements rejected by the 
content expert are automatically rejected in subsequent lists. 
The reason for this is that rejecting a single element firom an 
n-sized list will tend to eliminate the need to consider m/n 
elements in an accompanying m-sized list, since for each 
element in the m-sized list, the probability of getting 
removed from consideration is about 1/n. To see why the 
probability for removal is 1/n, consider the fact that each 
element in the m-sized list must appear in some content 
example with an element in the n-sized list. Without any 
other information, it is just as likely for the elements in one 
list to depend on any given element in another list. Therefore 
each element in the n-sized list will likely be tied to m/n 
elements in the m-sized list. (To be precise, the probability 
is lower than m/n, since an element in the m-sized list can 
be associated with more than one element in the n-sized list. 
But the same line of reasoning sdll holds true.) Hence 
automatic contextual filtering will tend to remove m/n items 
from the m-sized list for each rejection in the n-sized fist. To 
maximize the automatic filtering and minimize the decision 
making of the content expert, m should be kept large as 
possible and n as small as possible. And this is why the 
smallest element lists are presented for consideration first. 
Relevant Segment Marking 

Another feature of the present invention is the concept 
tagging system's ability to automatically determine which 
part of each element relates to the learning concept based on 
the search criteria that was fulfilled. Using another language 
database example, if the search criteria is "*te kudasai", then 
the part of the sentence that says "te kudasai" is automati- 
cally marked as the relevant segment. In cases where no 
useful search criteria can be found, the concept tagging 
system allows the content expert to pick example sentences 
out of all sentences in the database, and to specify manually 
which part of the sentence relates to the grammar principle. 

Adaptable Language Learning Array 

Another advantage of the present invention over the prior 
art is that students with various native languages can be 
instructed in a single target subject matter such as a foreign 
language, using one content database. Instructions, button 
captions, and other instructional text are automatically 
imported from the database into the nmning program in each 
student's native language. This is especially useful in appfi- 
cations for teaching ESL (English as a Second Language), 
where the students' native languages may vary. Because the 
system is individually configurable for each student, the 
system may be used to instruct any number of students with 
dififering native languages in different target languages or 
disciplines all while monitoring, recording and reporting the 
individual progress of each student. In this manner the 
system may be used to teach native French, Spanish and 
English speaking students any given language for which a 
database has been configured or they might each be taught 
a different language or discipline at the same time. 
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lodividual Learning Activities or Applications 

The actual activities presented to a student will vary 
widely with the subject matter being studied. Typically, 
introductory activities are used to present new concepts, 
followed by review activities and, ultimately, testing activi- 
ties. Other advanced activities may also be used to increase 
comprehension beyond the testing stage. Tliese activities are 
collectively known as learning activities. 

In the preferred embodiment, the system is used for 
foreign language training and is referred to as the Technol- 
ogy Assisted Language Learning (TALL) system. In this 
preferred embodiment, activities are used to introduce, 
review, test and enhance comprehension of grammar, 
vocabulary or phrase memorization concepts. 

Introductory Grammar Activities. These activities arc 
used in the new or first stage of the track. In the preferred 
embodiment, Grammar concepts are introduced using any or 
a specified combination of the following activities. 

Example Sentences — Guess the Rule: wherein a series of 
sentences is presented and the student is prompted to guess 
the rule of grammar common to the sentences. 

Grammar Explanation: wherein a straightforward expla- 
nation of a specific grammar principle is displayed. This is 
typically used with the guess the rule exercise to explain the 
correct answer. 

Repeat after Native Speaker: wherein a recording of a 
native speaker is played and the student is prompted to 
record and play back his own pronunciation the phrase for 
pronunciation purposes in order to increase familiarity with 
the sentence and grammar principle. 

Identify Example Sentence: wherein a series of sentences 
is presented audibly or visually and the student is prompted 
to select the sentence containing a specified grammar prin- 
ciple. 

Guided Translation: wherein multiple columns of words 
are presented to the student who is prompted to select a word 
from each column to form a foreign language equivalent of 
a specified native language phrase. 

The following grammar review activities may also be 
used to introduce new concepts when combined with the 
Grammar Explanation above. 

Grammar Review Activities. These activities are used in 
the review grammar stage of the preferred embodiment. 

Identify General Meaning; wherein a student selects a 
native language rough translation from a selection of target 
language phrases. 

Multiple Choice Completion: wherein a student is 
prompted to fill in a blank portion of a foreign language 
sentence from a selection of possible fillers, one of which 
best represents a grammar principle. 

Large Context Multiple Choice: similar to multiple choice 
completion above, but multiple blanks are displayed 
within an entire paragraph. 
Fill-in-the-Blank Completion: similar to multiple choice 
completion above, except the student is prompted to fill 
in a blank in a target language phrase by simply typing 
the correct word in its proper location without alterna- 
tive selections. The native language translation may be 
displayed to avoid ambiguity. 
Large Context Fill-in-the-Blank: similar to fill-in-the- 
blank completion above, but multiple blanks are dis- 
played within an entire paragraph. 
Entire Box Form Conjugation: wherein a verb is dis- 
played in the infinitive form and various blank boxes 
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are displayed into which the student is prompted to type 
the conjugated form of the verb. In Japanese, this 
activity can be used for the seven verb bases and the 
desu and masu boxes, 
s Single Box-Element Form Conjugation: similar to entire 
box form conjugation above, but with only a single 
box. The form desired to be input is denoted with a 
native language translation or a description of the form. 

Transform Sentence or Dehydrated Sentence: wherein a 
10 target language sentence is displayed and the student is 
prompted to modify the sentence tising a specific 
grammar principle. For example, the student may be 
prompted to change the verb tense. Multiple transfor- 
mations may also be used. 

Answer Questions Related to Grammar: wherein the 
student is prompted to answer a question about a target 
language phrase requiring knowledge of a grammar 
principle. The phrase may be presented in audible form 
or in textual form and the answer may be selected using 
a multiple choice formal or a type-in format. 

Listen for Grammar: A target language audio segment is 
played and the student is prompted to respond via 
mouse of keyboard when the specified grammar prin- 
ciple is played. The text of the audio segment is then 
displayed and the student is prompted to translate into 
her native language. A model translation is then shown 
so the student can assess her translation ability. 

Situational Grammar Production or Transfonmation: The 
student is prompted to perform a short task or series of 
3Q tasks in the target language wherein the student's voice 
is recorded in performing the task. A recording of a 
model response is then played. The student compares 
his recording to the model response to assess his 
performance. 

35 Grammar Test Activities. In the preferred embodiment, 
the content expert will select one or more of the above 
review activities for the test stage. 

Multiple Principle Grammar Activities. These activities 
are used outside the normal activity track to build student 

40 mastery of a concept. They involve multiple grammar prin- 
ciples. 

Sentence Builder: This activity involves a growing sen- 
tence which is added upon in each step. The activity 
displays a basic native language sentence with a prompt 

45 to translate (i.e., I eat). This step is then checked for 
accuracy and corrections are made when necessary. The 
next step adds to this initial sentence (i.e., I eat cake). 
The new response is checked for accuracy and so on 
until a complex sentence is completed. Student 

50 responses are formed from a selection list displayed on 
the screen. 

Question and Answer Sentence Builder: A question is 
asked in a target language and the student is prompted 
to respond by building a response from words and 
55 phrases displayed in a box on the screen. This response 
is then corrected and a follow-up question is asked with 
similar structure to the first, but with increased com- 
plexity. The process goes on until all phrases and words 
in the answer selection box are used. 
60 Open-Ended Grammar Activities. These activities are 
used outside the normal activity track to build student 
mastery of a concept. They involve the application of 
grammar principles to open-ended language production. 
Create Sentences: The student is prompted to create a 
65 number of sentences in the target language using sev- 
eral specified grammar principles. This response is sent 
to an instructor for feedback. 
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Practice What You Have Learned: A question situation is Phrase Test Activity. The progressive typing activity is 

presented in text form and the student is prompted to used as the test activity in the preferred embodiment, 

respond by typing. This situation and response are sent These activities may be used in any combination as 

to the instructor for review. designated by the instructional designer and the content 

Translate Sentence: A sentence is presented in text or 5 expert according to the needs of the student and the field of 

audio form in either a native or target language and the study. 

student is prompted to respond in the alternative Ian- p^^^j Learn ine System 
guage form, target or native language respectively. 

Introductory Vocabulary Activities. These activities are Another advance of the present invention over the prior 

used in the new or first stage of the track. In the preferred art is the employment of "pool" review. The Pool Learning 

embodiment, vocabulary words and phrases are introduced System provides for systematically spaced review based on 

using the following activity. the correctness and speed of each learner's individual, 

Hash Cards: The vocabulary item is displayed along with ^^^^^^^ ^^^^^^ responses. For example, in the language 

the native language translation which can be selectively applicaUon of the preferred embodiment, this technique is 

removed from display. Sentences known to the student ^PP^^^ ^° ^^^^^ grammar prmciples, phrases and 

are also used as examples of the vocabulary item. vocabulary. The same concepts apply to learning new ter- 

Audio files corresponding to the sentences and the mmology of a trade or profession and learning to perform 

vocabulary item are available for obtaining an audible mathematical operations, 

familiarity. The student is encouraged to deselect the Activity Tracks 

native language translation to enhance memorization. ^^^^ ^^^^ O-^- medical term, grammar concept, vocabu- 

Vocabulary Review Activities. These activities are used in ^°'^) ^^^^ ^ ^^^"^^^ P^^l Icaramg system 

the review vocabulary stage of the preferred embodiment. !^ '""^^ ^ database and assigned a sequence of 

XM r^u ' i-u • j- instructional activities. These sequences arc referred to as 

Multiple Choice: T^e student is given a word m audio or ^^j.^- ^^^^^ ^^^^^^^ ^ 

text format in the target or native language and learning content to the system, assigns activity tracks to each 

prompted to select its equivalent in the other language -^^^ ^^^^^^^ example, in a language application 

from a list of selections given m audio or text format. ^,,,bulary items may all share a single activity track. 

Typing m Context: A native language sentence is given phrases may share another single track. Grammar principles 

and the student is prompted to complete a target Ian- may have a number of separate tracks and may even have a 
guage translation of the sentence by filling in blanked 30 separate track for each principle or item. The reason for 

out words. ^sijig several distinct tracks for teaching grammar principles 

Typing out of Context: A native language word is given is that the skills they entail vary widely, compared to 

and the student is prompted to type the target language vocabulary and phrases. Other fields of smdy may follow a 

translation. simUar pattern. 
Vocabulary Test Activities. In the preferred embodiment, 35 The activity track specifies the sequence of activities used 

the typing out ofcontcxt activity is used for testing purposes. to introduce, review, and test the item. FIG. 6 shows 

Introductory Phrase Activities. These activities are used in example activity tracks wherein Track 1 may represent an 

the new or first stage of the track. In the preferred activity track for a vocabulary item and Track 2 may 

embodiment, phrases are introduced using the following represent an activity track for a grammar principle. Easy/ 
activity. 40 receptive activities occur at the beginning of the track, and 

Flash Cards: This activity is the same as the flash card difficult/productive activities occur toward the end. 

activity described in the vocabulary section above Generally, every activity in the track is as difiScult or more 

except phrases are used rather than single words. difBcult than the preceding activity. For example, the track 

Phrase Review Activities. These activities are used in the vocabulary items begins with a computerized flash card 
phrase review stage of the preferred embodiment, 45 exercise, moves to a multiple choice exercise and then to 

Multiple Choice: This activity is similar to the vocabulary ^^P^^S context and typing out of context at the end of the 

version described above except phrases arc used in lieu track, 

of words and additional combinations of prompt and Learning or Review Pools 

response options arc used (i.e., phrase with filler, phrase ^^^^ occurrence of an activity is directly related to a 

with example sentence), ieammg pool, or review pool. The term pool reflects the fact 

n •* * J * ' 1, • .t. that several items may be at the same point of progress in the 

Recite: The student is prompted to recite a phrase in the , . r ' 1 ^ * J *• 

^ , , . J ■ u . . track, and hence form a pool of items to draw upon for 

target language when prompted with a native languaee „„-^.„ * *• t-u ^ * * * 11 r 

u . 11*. r review or testing. The pool represents or contains all of the 

phrase. The correct answer IS then revealed m text form -.^ • *• c- 1 i 

^ , , J , . . J . . J. . .i. . items available for use in a speafic activity. Several pools 

and the student is prompted to indicate the correctness accessible to a student, but each may be acce^ible 

of the response via pomUng device or keyboard. ^^^^^^^ ^ ^^^^^^^ ^^^^^^^ 

Order Parts: A target language phrase is divided into Stages 

several parts and the student is prompted to assemble The pools of a track arc normaUy grouped into three 

the parts. A native language translation is provided to stages: new, review and test as shown in FIG. 1. In the new 

indicate the desired response. This acUvily utilizes a stage 610 or 640, items are introduced one at a time. In a 

novel prompting process which is further explained in language training application example, the flow of activities 

the section on Interactive Phrase Memorization. the new stage may run something like this: (1) give 

ProgressiveTyping: A native language phrase is displayed example sentences that help the student discover how the 

and the student is prompted to type the target language item is used in real language, (2) provide a concise definition 
translation. Hints are given in a novel process further 65 or explanation of how the item can be applied in general, (3) 

explained in the section on Interactive Phrase Memo- invite the student to practice using the item via several 

rization. simple activities that require some understanding of the 
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item. In order to establish an initial firm fouadation, the working group: (1) the student is struggling so much with 

student is typically allowed and encouraged to advance the item that an easier activity would be more appropriate, 

through all the pools in the new stage within a single study (2) the student has responded well enough to meet the 

session. minimum criteria for advancing the item to a higher pool. If 

In the review stage 620 or 650, a pool of items are studied 5 neither of these criteria arc fulfilled, the item is simply held 

together. All the items that are ready for a particular activity working group for further review. Refer to FIG. 7 for 

are brought together to form a working group. Questions ^® specific steps of the process. 

about these items are randomly presented to the student in '^^ determine whether the item is ready for advancement, 

the format of the current activity. After the student has many criteria may be taken into account. In the preferred 

reviewed all of the items ready for this activity, the next lO f "^^odiment any of four specific criteria are typically used: 

activity is chosen and so the cycle begins again. A feature of 1^^^ ."^"^^er of correct responses to questions about the item 

r,. * • *• • . V ..L, m the current review session, the recent number of correct 

this poruon of the present mvenUon is the ability to hmit the ^^^^ ^ g^^^ 

students exposure to a given Item to a single type of activity (askiming that the first response was correct), and the 

per day. For example, m one day a student may review items ^^^^^^ ^^^^^^ responses. All of these factors (the number 

K, L, and M with activity 14, then items Q, R, S, and T with 15 correct, percent correct, etc.) depend only on the student's 

activity 12, and items H, I, and J with activity 75. But, the performance in the current review session, for the item in 

student must wait until the next day to study items K, L, and question. In other words, only questions since the formation 

M with the activity that follows activity 14. The same is tme of the current working group are taken into account, llie 

of Q, R, S, T, H, I and J. Thus, the review process is instructional designer, a person who selects and modifies 

systematically spaced over time to ensure the item is 20 system variables to optimize instruction, specifies which of 

retained beyond short term memory. the four criteria shown at 710, 712, 714, and 716 in FIG. 7 

In the test stage 630 or 660, all the items ready for testing are important to achieve mastery of each activity. Since each 
are presented to the student one by one, for as long as the activity is distinct instructionally, their sets of advancement 
session lasts. Rather than recycling questions about each criteria also tend to differ. For example, response time 
item via a working group, the student is simply given one 25 (speed) may be important for a multiple-choice fonnat 
opportunity to respond appropriately to each item. Also, conjugation activity, but may not be for an open-ended 
there is a delay of several days between the review stage and sentence creation activity. The instructional designer also 
test stage. Asking one question per item and using a delay specifies the cutoff criterion for each element designated as 
between review and testing of an item allows the user of the important shown at 720, 722, 724, and 726. For example, if 
present invention to assess whether the student has retained 30 both the number of correct responses in-a-row and the 
the item in mid-tenn or long-term memory. The format of percent correct are important, then two numbers must be 
the question is determined by the item's current activity, specified, such as 2 correct*ia-a-row and 70% correct over- 
specified in the test stage of the track. If the student answers all. Generally, these criterion numbers will dififer for each 
a test question correctly, that item is considered mastered. If activity. To progress beyond the current activity 740, the 
missed, the item regresses to the review stage. 35 system may be configured such that the student must pass all 
Working Group of the selected criteria, not just one of them. 

The working group consists of all the items (medical If found lacking in a combination of the criteria for 
terms, grammar principles, vocabulary, etc.) that are cur- progressing, the student's performance may be compared 
rently being presented to the student by the program for with the criteria for regressing 730 which is similar to the 
review. As items are successfully reviewed and leave the 40 progression criteria. The number of wrong responses to an 
working group, new items take their place from a current item in the current review session may, for example, be the 
pool, so that the number of items in the working group criteria for regressing. As with the other criteria, the instruc- 
remains constant. The size of the working group depends on tional designer controls this number. If this number is 
which activity is being used for the review, as well as the exceeded, the item will regress 760. The purpose of regress- 
item type being studied. This is important since each activity 45 ing to an easier activity is to avoid causing the under- 
and item type place different mental demands upon the prepared student undo anxiety and frustration. The current 
student. For example, the working group size may be 7 when item simply continues in the working group for further 
reviewing vocabulary with the muhiple choice activity, and review 750 if neither the criteria for progressing nor the 
the working group size may be 2 when reviewing verb criteria for regressing are fulfilled, 
conjugation patterns which require the student to conjugate 50 Pool Advancement 

a verb in a given tense. The present invention allows the Once it has been determined that an item should progress 

instructor to adapt the working group size to the subject from the working pool, the present invention further deter- 

matter and the student's retention patterns. mines the extent of pool advancement. When a student has 

Pool Progression and Regression mastered a concept associated with an item or is more 

During the course of an activity the present invention 55 familiar with a concept, the student's time is better spent 

analyzes the student's responses to determine how well the studying other concepts. Therefore, the present invention 

student knows the concepts associated with a given item. measures the student's familiarity with a concept by the 

When a student responds to a question about an item the student's response characteristics. Any of the factors used to 

present invention will progress that item to a more advanced determine whether an item should progress can be used to 

pool for future review, hold that item in the present working 60 determine the extent of pool advancement. Other factors can 

pool for further study during the present session, or regress also be used. Typically, in the preferred embodiment, the 

the item to a previous pool for study in a less advanced percentage of correct responses, the time taken for the first 

activity. In this way the system sorts the items into different response, if correct, in relation to the student's dynamic 

review pools. After every response from the student, the average response time are used to determine the extent of 

present invention determines whether the item to which the 65 pool advancement. The dynamic average response time is a 

student just responded is ready to leave the working group. variable calculated by the present invention for use in 

Note that there are two reasons an item may leave the evaluating a student's comprehension level. 
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Dynamic Average Re^onse Time 

The dynamic average respoase time is recalculated with 
every correct response. Each student has a unique dynamic 
average response time for each item type. The process by 
which the dynamic average response time is calculated is 
shown in FIG. 8. Each student starts out with an initial 
average response time for each activity, as specified by the 
instructional designer. Initially the total number of responses 
tracked is zero. With each new correct response, the average 
response time is recalculated, and the number of responses 
tracked increments. After many responses are taken into 
account, the response time average tends to stabilize. In 
order to control how quickly the average can change after it 
stabilizes, the instructional designer specifies a minimum 
contribution percentage. For example, a vocabulary 
multiple -choice activity may have a minimum contribution 
percentage of 1%. This means that with each new correct 
response within this activity, the new average is 99% the old 
average response time and 1% the current response time. As 
shown in FIG. 5, the present invention first determines 
whether the number of responses is greater than or equal to 
the quotient of 100 divided by the minimum contribution 
percentage 810. If the number of responses does not yet 
equal this value the contribution percentage will be set to the 
value of 100 divided by the sum of the total responses 
tracked and one 830. If the total number of responses is 
greater than or equal to the quotient calculated at 810 the 
contribution percentage will be set to a value equal to the 
minimum contribution percentage 820. The new average 
response time will then be calculated as the sum of two 
products — the contribution percentage times the current 
response time, plus 100 minus the contribution percentage 
times the old average response time — divided by 100 (see 
equation, FIG. 8) 840. 

This technique of calculating a dynamic average response 
time is effective because: (1) it converges quickly to the 
individual student's response time initially, and then 
changes more slowly as information about the student's 
response time is acquired, and (2) even after many response 
times have been averaged, new response times hold some 
weight instead of becoming infinitely small, so that the 
calculated response time continues to converge nicely to the 
student's developing response time. 
Real-Time Response Evaluation for Pool Progression 

Using the above noted response criteria, the present 
invention determines the student's level of comprehension 
and familiarity with a concept and the associated extent of 
pool progression of a particular item using a "real-time 
response evaluation" procedure as illustrated in FIG. 9. The 
systems and methods of the present invention first determine 
whether all of the student's responses were correct 940. If 
not the item will advance only one pool 910. If all responses 
were correct the item will advance at least two pools 920. If 
the instructional designer has indicated that the time taken 
for the first response is important, 950 then a comparison of 
the student's first response time and the student's dynamic 
average response time as calculated according to FIG. 5 is 
performed 960. If the student's first response time is greater 
than the dynamic average response time, the item will 
advance two pools. If the student's first response time is less 
than the dynamic average response time, the item will 
advance three pools. This is the default procedure, and has 
been found to work well with language training applications, 
however the instructional designer may specify other criteria 
for pool advancement for each activity and each item type. 
Working Group Dynamics 

To further illustrate the progression of items through the 
pool system and the dynamics of the working group, FIG. 10 
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shows a working group operating with three items. Ques- 
tions about each item are presented to the student in a 
random sequence, so that the student cannot anticipate 
which question will come next. This random ordering 

5 encourages the student to pay close attention to each item in 
the working group, since the next question could require 
knowledge of any item. Moreover, if questions always 
followed a fixed sequence, the student would inadvertently 
learn to depend on the sequence itself in order to remember 

10 the correct response. This problem is prevented by random 
ordering. The items in the working group are reshuffled after 
asking one question for each item. This pattern of reshuffling 
is illustrated in FIG. 10 after questions 3 and 6. Notice that 
upon filling the woricing group, the order of questions is 

15 ACB. After shufiOing, the sequence of the next cycle is BDA. 
Then another shuffle occurs, resulting in the sequence DBA. 
Cycling through all the items in the working group in this 
manner ensures that all items are given equal opportimity to 
be presented throughout the review session. 

20 FIG. 10 also illustrates the dynamic nature of pool pro- 
gression wherein item C leaves the working group being 
replaced by item D when question 2 (about item C) was 
answered with a fast and correct response. After an incorrect 
response to question 3 (about item B), item B stays within 

25 the working group. After question 3 the working group is 
exhausted and is therefore reshuffled before further ques- 
tioning. Question 4 (about item B) is also answered incor- 
rectly so item B again remains in the working group. 
Question 5 (about item D) is then asked and receives a slow 

30 but correct response causing it to remain in the working 
group. Question 6 (about item A) also receives a slow and 
correct response and A stays in the working group after 
which the working group is again exhausted and reshuffled. 
Question 7 is posed and receives a second correct and slow 

35 response which causes item D to progress from the working 
group cither because the number correct is now at or above 
the minimum or the number correct-in-a-row is at or above 
the minimum number. Question 8, the third question about 
item B, is answered incorrectly demonstrating the student's 

40 lack of comprehension of the concept related to item B. Item 
B, therefore, regresses, leaving the working group, to the 
review pool associated with the next easiest activity. Ques- 
tion 9 (about item A) receives a correct and slow response 
causing it to leave the working group, however, it advances 

45 only one pool because question 1 regarding item A was 
previously answered incorrectly. As each item progresses 
from the working group, another item takes its place from 
the list of items ready for that activity. 
The total number of questions about a particular item in 

50 a review session depends on how well the student responds 
to the questions. In FIG, 10, for example, the student 
responds quickly and correctly to item C on the first attempt, 
which means that no more questions will be asked about 
item C in the current session (see question 2). It will appear 

55 three days later in another review session, with a more 
advanced activity. The student struggles with item B, on the 
other hand, answering incorrectly three times. The third 
wrong response causes item B to leave the working group 
and regress to a review pool associated with an easier 

60 activity (see question 8). 

FIG. 6 further demonstrates the dynamic progression and 
advancement in the pool system and the application of 
systematically-spaced review to learning items A, B, C and 
D. Items which would replace tliese items as they progress 

65 from the working group are not shown to simplify the 
diagram. On day 1, all items are studied in new pool 1, 
making them available for review pool 1 after a one-day 
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delay. On day 2, all items are studied in review pool 1, but 11 also illustrates how learning progress varies for a student 

this time the student responds especially well to the ques- learning items A, B, C and D. The hypothetical student 

tions involving item C. This results in item C jumping from represented in FIG. U has advanced item B from the new 

review pool 1 to review pool 3, entirely skipping review stage to the test stage after 14 days, whereas item C is ready 

pool 2. Notice that this also means that item C is delayed 2 5 for testing after 9 days. This figure also illustrates the fact 

days instead of 1. The instructional principles that suggest that more time is spent learning items that arc more chal- 

lengthening the delay are: (1) when the student appears to lenging to the student. For example, item B is encountered 

know an item particularly well, he/she will probably re mem- 12 dififerent times before the test, while item C is only 

ber it longer than other items, and (2) less time should be encountered 4 times before the test, 

spent on items that are easy for the student, to allow more lO Systematically Spaced Review 

time for challenging items. Some students employ a study strategy wherein they 

Items A and D also progress from review pool 1, but only study intensely just prior to testing in order to commit a great 

advance to the next pool. Item B, however, remains in the deal of information to short term memory immediately 

same pool without progression or regression and will be before the test. This technique is called cramming and leads 

available for the same activity the next day. i5 to lower retention when the short term memory period 

On day 3, Items A and D are available in review pool 2 expires. The pool learning system of the present invention 

while item B is available in review pool 1. Higher review ensures that items are reviewed periodically over several 

pool numbers typically reflect more advanced, progressively days or weeks. This is accomplished by putting a delay 

diflBcult activities so items A and D may be reviewed in a between each pool. Typically a one-day delay occurs 

way so as to increase the student's understanding of the 20 between each review pool. A several-day delay occurs just 

associated concepts beyond the level in review pool 1 while before being tested on an item. Without these delays in force, 

item B will be presented in an easier activity to instill a basic a student might choose to study and progress an item from 

knowledge in the student. As reflected in FIG. 6, the student the new stage to the test stage in a single day resulting in 

responds positively to items A and D advancing them by two short term memory retention only. The present invention, 

pools while item B advances again by only one pool. 25 therefore, employs systematically spaced review and testing 

On day 4, only items B and C will be available. Item B to ensure that items should always be in mid- or long-term 

will be available in review pool 2 and item C will again be memory by the time they pass the test, 
available, after its one day delay, in review pool 3 where it 

will typicaUy be presented in a more advanced activity than Dynamic Menu System 

that of review pool 2. Some questions regarding item C are 30 , , . . , 

answered correctly such that it advaoces 2 pools and some '° " learning session, the student must choose from a 

of the questions regarding item B are answered correctly °^ '="."^"'17 f^'e^^'"^' r"''^^^,' ' 

such that it advances one pool. language training example such as the preferred 

On day 5, item B is available, now in review pool 3. and embodiment, the student may have up to three program areas 

itemsA and Dare available in review pool 4. This time item 35 '° S''""""' vocabulary or phrases. All 

B advances one pool as the student continues to be chal- P'^S™" Yf."' P°"="''.'Jly '^''^ "l^^' ' 

lenged by its associated concept, item D fails to progress and f °"'P.°°«'^ ^'^^^^^.u °'™"' °^ """^ menu choices for 

remains in the same pool due to poor student response, and ^ems with the pool learmng systetn. These would 

item A advances two pools giving the student a one day ?f ^T", ^'^T^'.' Review Grammar, Test Granjmar, New 

break from that concept on the following day. 40 Vocabulary, Review Vocabulary, Test Vocabulary, New 

On day 6, the question regarding item C is answered Phr»^^ Review Phrases, and Test Phrases. Only programs 

quickly and correctly and item C advances directly to the test ^'"'f'' ^ ^ufBcient number of leammg items are available 

pool where, after a 3 day delay, it wiU be accessible to the ^""'y """"^fi. °" '"^^ dynamic menu, therefore all 

student for final testing. Item B advances one pool and item <=?°|«f 1"^^ be displayed at any given tmie. Once the 

D advances two pools 45 "^^^ selected a program m the menu, the program 

On day 7, item A is available again and after a positive ".'S'"^ process of choosing which activities to 

student response advances to the test pool. Item B. still f to the student. For the new and test programs, this may 

challenging the student, regresses back to pool 4 (the pool it s»aightfonvard. For the new programs, the actmly 

occupied the previous day) where it becomes immediately '^^^ P"""^/ ^''"^"'^f ^''.'^ .instructional 

available the same day in a less challenging activity. 50 ^ner or the student may be given a cho.ce among a 

On the following days similar progression takes place f ^lecUon ot mtrodiictory activiUes. For the test programs, 

until all items progress to the test pool. TTie three day delay activity used to test a given item is typicaUy assigned by 

before testing is crucial to ensuring that the student has wstracUonal designer or content expert. Hie selecUon of 

committed the learned information to medium or long terai '^view acUvities is more complex and arrives at a sequence, 

mf^mnr^r cc determined by the present mvention, which maximizes the 

Integrated Concept/Student Adaptation ? efficiency using student-specific activity 

Another feature of the present invention is its ability to sequencing, 

automatically tailor learning sessions to the individual stu- Student-Specific Activity Sequencing 

dent. This is referred to as integrated concept/student adap- / ^ & 

tation. Individualized learning has more potential for learn- 60 FIG. 12 shows the method by which the present invention 

ing efficiency than group-paced learning. With the pool selects review activities. To determine the next review 

learning system, each student learns each individual item at activity, the present invention queries the user's database to 

his/her own pace, for that particular item. This process find the set of activities for which there are any items ready 

avoids static lessons with their associated boredom and for review 1210. lliese activities are ordered by the date of 

ineflSciency. For example one student might master items X 65 availability. For example, activities with items which have 

and Y quickly and item Z slowly, while another student been available the longest are given first priority. An item 

might master item Y quickly, and items X and Z slowly. FIG. becomes available when it enters a pool that is assigned to 
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a given activity and the date previously targeted for study of 
that item has passed. For example, when a student responds 
correctly to a question in the new stage, the item is advanced 
to the first pool in the review stage where it waits until the 
activity assigned to that pool is used. Once these activities s 
arc identified and ordered the system and method of the 
present invention set an incrementing variable "i" 1215 to 
the value of one to keep track of activity priority. This 
variable will be incremented to 2 and so on as activities are 
rejected 1245. The new system and methods of the present lO 
invention then begin an activity eligibility analysis by que- 
rying the database for information on the activity with first 
priority. This information includes the ideal number of items 
to use with the activity, the minimum number of items to be 
used with the activity as well as other information desig- 15 
nated by the instmctional designer 1220. The invention then 
finds the number of items available for use with the activity 
1225 and checks to see if the ideal number of items is 
available 1230. If the ideal number of items is available the 
activity will be presented without further analysis 1235, if 20 
the ideal number is not available a further check is made to 
see if the activity is urgent 1255. The urgency limit can be 
defined by the instructional designer to be a given amount of 
time that the item has been available. If an item has been 
available for a period of time in excess of the urgency limit 25 
its activity becomes urgent. When an activity is urgent the 
present invention will search for additional items that have 
previously advanced past its pool and add those items to its 
pool 1260. The present invention will then check to see if the 
minimum number of items is available to present the activity 30 
1265. If the number of items available for the activity is less 
than the minimum number required (a number specified by 
the content expert), the program will increment to the 
activity with the next highest priority 1240 and begin the 
activity eligibility analysis anew 1220. When all activities 35 
with items ready for review have been analyzed, the session 
is ended 1250. This cycle continues until a review activity is 
chosen, or until the set of reviewable activities is exhausted. 

Once ao activity is chosen for review, the student contin- 
ues to study with it until an ideal number of items have 40 
advanced out of the working group or the current pool is 
exhausted. Then another activity is chosen using the process 
in FTG. 12, as explained above. 

Student-Specific Concept Filtering 

Another feature of the present invention is student- 
specific concept filtering, the ability to select database 
content based on an individual student's relative progress in 
relation to the difificulty of concepts contained in the data- 
base elements. Concepts and principles are presented to the so 
student in order of increasing diJOSculty and complexity. As 
a student is introduced to new concepts, the program keeps 
track of the concepts to which the student is exposed. The 
program can then limit the content to which a student is 
exposed in the various activities to concepts that are at or 55 
near the student's level of progress. This feature may best be 
demonstrated in another language application example as it 
is used in the preferred embodiment. When a user studies the 
Japanese grammar principle Bte+kudasai tbe program per- 
forms a search based on that grammar principle and finds 60 
example sentences that are tagged as examples of that 
principle via the concept tagging process described above. 
However, some of those sample sentences may not be 
appropriate for the user, that is, they may contain other 
complicated principles that will overwhelm and confuse the 65 
user. Instead of showing all the candidate sentences, the 
program will filter out the complicated examples by com- 



paring the grammar principles and vocabulary contained in 
the sentences with the grammar principles and vocabulary 
previously studied by the learner. Through this method the 
sample sentences are taUored to each individual's prior 
experience and the learner is challenged but not over- 
whelmed. The details and logic of this process are shown in 
FIG. 13. 

Once all instances of grammar in the sentences have been 
recorded in a database using the concept tagging process, 
effective example sentences (content examples) can be cho- 
sen for grammar activities. Rather than just choose any 
example sentence that uses the grammar principle being 
taught, sentences are selected which are most likely to be 
understood by the student. 

Once the present invention has determined the number of 
example sentences needed for the current activity 1305, all 
sentences that contain an example of the selected grammar 
principle and which do not have less than the maximum 
number of grammar principles that the student has not yet 
studied are drawn out of the content database 1310 for use. 
Then, the grammar principles contained in those sentences 
are compared to a list of grammar principles already studied 
by the student to determine the number of grammar prin- 
ciples the sentence contains that the student has not yet 
studied 1320. A set of those sentences with the least 
instances of grammar not studied by the student is compiled 
and saved for the next filtering step. In the next step, the 
vocabulary contained in the filtered sentences is then com- 
pared with the vocabulary studied by the smdent to deter- 
mine which sentences contain the most instances of vocabu- 
lary not yet studied by the student 1330. ITie sentences 
coming out of this step with the fewest instances of vocabu- 
lary not yet studied by the student are then used as example 
content in an activity addressing the grammar principle 
being introduced or reviewed 1340. 

The instructional designer can control the balance of 
grammar filtering versus vocabulary filtering by specifying 
the relative size of sets B and C. If the ratio is set to 2, half 
of the sentences in set B will be chosen for set C, based on 
how much not-yet-learned vocabulary the sentences contain. 
If the ratio is set to 4, then only one-fourth of the sentences 
in B would be allowed in C. 

Create Module 

Another novel aspect of the present invention is the 
Create module. By using the Create module of the TAL 
software the learner has an opportunity to apply newly 
learned skills to a whole, creative activity. Users can make 
print materials, access and manipulate the text from the 
database, and make custom audio recordings from the audio 
files stored on the computer. The user has the ability to create 
customized study materials from the elements in the present 
invention's databases. Students can create textual study 
materials that can be imported into a word processor for 
manipulation and exercise or print directly from the program 
to create memorization lists or other materials. Each of the 
database elements also has a corresponding audio file. The 
user can listen to the audio files directly from the computer 
or arrange the files to make an audio recording of their 
selection. 

Interactive Phrase Memorization 

Yet another feature of the present invention is a group of 
activities designed to help the student memorize database 
elements or learning items. This feature is extremely useful 
in learning new terminology for a new profession such as the 
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medical or nursing fields as well as learning phrases of a new 
language. To produce speech in a target language quickly, a 
student memorizes useful sentence patterns and common 
expressions. Several activities within the present invention 
help the student to memorize these phrases efiacienlly and s 
comprise the interactive phrase memorization system. These 
activities include "order parts" and "progressive typing." All 
interactive phrase memorization activities are spaced and 
coordinated via the pool learning system described previ- 
ously. Although these activities are applied to learning a lo 
second language in the preferred embodiment, the same 
activities could be applied in any situation that involves 
memorization of sentences, sequences of items or related or 
associated terms. Also note that these activities apply to the 
memorization of long passages of text as weU as short 15 
sentence fragments. 
Order Parts Activity 

The order parts activity requires the smdent to take a 
scrambled list of words of a sentence or phrase parts, and 
rearrange them to reconstruct the original word structure. 20 
The scrambled list may be in any order that docs not hint at 
the original order of the sentence or phrase. An alphabetical 
order may be effective for most applications. The number of 
words in each part decreases until the student can recreate 
the sentence one word at a time. To prepare the first 25 
order-parts exercise for a given sentence, the sentence is 
broken up into initial parts via the process shown in FIG. 14. 

The purpose of this process is to divide the sentence into 
phrase parts that are individually meaoingful. The process 
begins by dividing the sentence or phrase into "initial 30 
segment candidates" (ISC's) which must be at least a 
minimum character length and must be divided by thought 
separators such as spaces, dashes, punctuation symbols, etc 
1410. After an ISC is selected in this marmer from a given 
sentence, it is evaluated to see if it exceeds a maximum 35 
length 1420. If is does not exceed the maximum length it 
becomes a segment and the next ISC is found and evaluated, 
if it exceeds the maximum length it will undergo further 
analysis and modification. If the ISC exceeds the maximum 
length and is the first segment of a sentence or phrase 1430 40 
it will be split in half before proceeding to the next ISC; 
otherwise it is further checked for a thought separator in the 
first half of the ISC 1440. If it does not have a thought 
separator in the first half, the ISC will be split in half and the 
next ISC will be evaluated 1490. If the ISC has a thought 45 
separator in the first half, the ISC is divided at the thought 
separator and the portion preceding the thought separator is 
tentatively added to the previous segment 1460 after which 
the previous segment is again evaluated to see if it exceeds 
the maximum length with its newly added portion 1470. If 50 
the previous segment now exceeds the maximum length, the 
added portion will be removed and given back to the current 
ISC 1480. The current ISC wiU, then, be split in half 1490. 
If the previous segment does not exceed the maximum 
length with its newly added portion, it will retain the newly 55 
added portion and the current, newly shortened, ISC will be 
rechecked for length 1450, if it still exceeds the maximum 
length it will be split in half 1490 otherwise it will become 
the next segment. This pnDcess is repealed until no text is left 
in the sentence 1494. 60 

After this segment division process any punctuation or 
capitalization at the beginning or end of each segment is 
removed. This makes ordering the parts more challenging, 
since otherwise the student could identify the first part by its 
capitalization and the last part by its punctuation, such as a 65 
period. The parts are arranged in a scrambled list and 
presented to the student for sequencing. In the preferred 
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embodiment, the scrambled list takes the form of alphabeti- 
cal ordering of parts. The native language translation or 
explanation of the phrase is given, and the student is invited 
to reconstruct the equivalent target subject matter word 
structure by clicking on the listed parts in the right order. If 
the student succeeds, then the next exercise is made more 
challenging by subdividing the initial parts. If the student 
does not succeed, then the next exercise is made easier by 
combining the initial parts to form larger parts. 

This process of subdividing or combining occurs after 
every success or failure. FIG. 15 shows the process for 
determining how diflBcult or easy to make the next question. 
When a student answers a step conectly 1505, the present 
invention checks to see if the student has already progressed 
to the most complex step (where the content example has 
been divided into individual words and distractors have been 
added) 1540. If this final step has been reached the activity 
is done 1545, otherwise it is next determined if the student 
has answered any step in the activity incorrectly 1550. If any 
step has been answered incorrectly the activity advances one 
step 1555, if all steps have been answered correctly the 
activity will advance two steps 1560. When a step is 
answered incorrectly 1505, the present invention determines 
whether the student has already relapsed to the simplest step 
of the activity where the content example is divided into 
only two parts 1510. If the student is already at the simplest 
step, the step is simply repeated 1515 until it is answered 
correctly. Otherwise, the answer is evaluated to determine if 
it was almost correct 1520, and, if so, the step will be 
repeated 1515. Whether a response is "almost correct" is 
determined by the percentage of words that are placed in the 
correct order. The instructional designer may change this 
percentage to fit the circumstances. In the preferred embodi- 
ment a percentage of 85% is commonly used for language 
training. If the answer is not almost correct, the present 
invention next determines if the student has answered any 
previous steps in the activity incorrectly 1525. If previous 
steps have been answered incorrectly the activity will 
regress two steps 1530, If all previous steps were answered 
correctly the activity will regress only one step 1535. 

If the student continues to struggle with every question, 
the simplest case will ultimately be reached, which merely 
requires the student to order two parts, the first and last 
halves of the phrase. After answering incorrectly, the student 
is shown the correct answer, and the discrepancies in his/her 
answer are highlighted. With every question, the phrase 
becomes more entrenched in the student's memory. Soon the 
student is able to respond correctly to the most difficult 
order-parts question, which requires the student to find the 
individual words of the phrase among a list of distractors. 
Typically, the steps will be made progressively simpler by 
recursively doubling the size of the parts. In other words, the 
number of ISC's in each part is 1, 2, 4, 8, and so on up until 
there are only two parts to be sequenced. To make the steps 
progressively more difficult, the ISC's are divided into 
fractions V4, V3, Vi, Vis, Vs, Via, Vie, V24, V32, and so on until 
each part only contains one word. The final step adds 
distractors to the list of single-word parts, so that the student 
is required to recognize which words are in the phrase, as 
well as how they are ordered. 

The order parts activity is effective because it helps the 
student to link phrase fragments in the right sequence. 
Linking phrase or sentence fragments is essential to memo- 
rizing verbal sequences. By adapting to the student *s ability, 
the activity challenges but does not overwhelm the student. 
For this reason, students that find memorization dJfi&cult will 
especially benefit from the order parts activity. The simplest 



03/16/2004, EAST version: 1.4.1 



6,077,085 

25 26 

question only requires the student to recall the sequence of After making the second attempt, the hints are decreased 

two items, the two halves of the phrase. Even the strugghng or increased for subsequent steps according to the following 

learner will have hide trouble answering this question pattern. If a particular word is missed again despite the hints, 

correcUy. As the student succeeds at one step, the next step then the number of hint characters is increased by the 

requires them to recall more and more sequential links. 5 number of words missed this time divided by five. Eventu- 

When the student cannot recaU one of the sequential links, ^^^^^^ characters is constrained by the 

he/she may still be able to arnve at the correct answer by length of the word. Almost, all letters in the word except the 

comparing each part to see which one best foUows the ^^^^ y^^,^^ ^^^^^j^^ jf ^^^^^^ ^^^^^j ^^^^^^ ^ ^^^^ 

previous part gramma ically or senaantically. By doing this, ^^^^ previously missed, then the number of hint char- 

the student partiaUy relies on knowledge other than memory , t ,u * i- u 1 j ^ * c * n 

of the phrase, however the mental effort expended by acters for that word is halved at the nc)rt ^ 

considering the candidates will contribute to making the hmt characters will be given for that word. Ihis process 

sentence parts more memorable. continues until the student can produce the phrase without 

The pool learning system coordinates all the phrase ^^^^ characters showing, 
memorization activities aUowing several phrases to be Individualized Automated Student Monitoring and Report- 
reviewed with the order parts activity at once. This makes 15 ing 

remembering the phrase from one question to the next more Additionally, the present invention includes a method for 

challenging, since the student must juggle several in his/her generating customized reports regarding the progress and 

mind simuUaneously. The order parts activity may also span study history of each student. Teachers can access computer 

several days for a given phrase to further challenge the generated student reports which give information about how 

student. 20 the student is spending her time, which topics are mastered, 

Progressive Typing Activity and which topics she finds difficult. Teachers and adminis- 

Another novel memorization and learning activity of the trators can follow up by giving appropriate feedback and 

present invention is the progressive-typing activity. It is guidance. As a student studies, the present invention moni- 

more challenging than the order-parts activity, since it tors and records the time spent on each activity, items 

requires the student to produce the phrase character by 25 studied, items mastered, amount of idle time, and other 

character. Ultimately, this activity teaches the student to variables. This data is recorded and reported in a format 

recall the entire phrase verbatim without any hints other than which may be configured by the instructional designer. A 

the native language translation or explanation of the term. sample report is included below. 

What makes this activity noteworthy, however, is its ability The present invention may be embodied in other specific 

to carefully work up to the ultimate step by generating 30 forms without departing from its spirit or essential charac- 

questions that are individualized to the studenl*s abUity to leristics. The described embodiments are to be considered in 

recall the given phrase. all respects only as illustrative and not restrictive. The scope 

The present invention adapts each progressive typing of the invention is, therefore, indicated by the appended 

question to the individual learner thereby enhancing the claims rather than by the foregoing description. All changes 

learning experience. Before reaching this activity, the stu- 35 which come within the meaning and range of equivalency of 

dent has become quite familiar witb the phrase via the other the claims are to be embraced within their scope, 
memorization activities, such as the order-parts activity. 

This preparation is vital to prevent the progressive-typing Sample Report 

activity from being too difiScult, The very first step of the John R. Doe 

progressive-typing activity is called the "challenge" step. 40 Report covers 12 sessions and 7 days (Mar. 24, 1997 7:04:59 

The "challenge" step invites the student to show how well AM to Mar. 31, 1997 1:34:36 PM) 

he/she can recall the phrase, by typing it without any hints Tasks 

other than the native language translation. This step is Average time spent since last report: 38 minutes/session, 

necessary to assess which parts of the phrase the student has . , , , > 1 . 1 

, J / . . u . / . . .i J . Total worked on: 41 tasks, 

already learned, so that subsequent steps can be tailored to 45 pj^-^g^g 

his/her ability. The student is not expected to recall the entire 

phrase at this point-thc quesUon is presented in such as ^^^^^S^ ^"^^ ^P^"^ ^^^^ ^«P«^t= ^ minutes/session, 

way as to appear not as a test, but as a challenge to see what Mastered smce last report: 6 phrases, 

they already know. After this attempt, and every subsequent List of phrases mastered since last report: 

attempt, the student is shown the correct answer and the 50 Ten no OtAsama. 

parts he/she missed are highlighted. Nanpaaeji desu ka? 

For the second step, the words that the student got right Hai, wakarimasu. 

are pre- filled in the answer box, so that they do not need to Konnichiwa. 

be retyped. Also, for each word that was raLssed, a few 0-genki desu ka? 

characters are revealed as hints, and the remaining charac- 5S Sayonara. 

ters are represented by dashes. For example, seeing the first Xolal mastered: 6 phrases, 

few characters or the last few characters of a missed word is Grammar 

a powerful aid for recalling the entire word. The student is ^ ^^^^ j^^^ 22 minutes/session, 

also helped by seeing the number of letters m each word. , . , . . ... 

The number of hint characters revealed depends on the total 60 ^^^'"^ ^"P°^^^ ^ pnnciples. 

number of words missed. For foreign language training, one mastered: 12 pnnciples. 

hint character for every five words missed has proven to be Vocabulary 

an effective ratio, however, this number can be adjusted for Average time spent since last report: 7 minutes/session, 

other apphcations. Accordingly, if two words were missed. Mastered since last report: 45 words, 

then the first character of both is revealed. If ten words were 65 Total mastered: 252 words, 

missed, then the first two characters of each of the ten words Listening Comprehension 

are shown, and so on. Average time spent since last report: 0 minutes/session. 
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Scores on tests taken so far: 7/19 4/19 7/18 8/13 7/19 6/19. 
Create 

Average time speat since last report: 0 minutes/session. 
Summary 

Total of average time spent since last report; 74 minutes/ 
session. 

Portion of above total w^hich is mostly *idle* time: 28 

minutes/session. 
Percent of above total which is mostly 'idle* time: 37% 
Times learner recorded with the microphone since last 

report: 24 recordings/session. 
Number of practices with teacher since last report: 2 
Total time spent in practices with teacher since last report: 

19 minutes. 
What is claimed is: 

1. A technology assisted learning system comprising: 

at least one computer comprising a visual display device, 

a central processing unit (CPU), one or more input 

devices, one or more output devices, and one or more 

memory portions; 
said memory portion(s) comprising: 

means for building and maintaining a computerized 
database comprised of database elements to be 
learned and their characteristics; 

means for conducting learning activities that will 
access said database for learning content; 

means for tagging said database elements to identify 
them as containing specified conceptual principles 
comprising a concept tagging system; 

means for presenting said elements using systemati- 
cally spaced review and testing in a pool learning 
system; 

means for providing selective access to said learning 
activities using student specific activity sequencing; 

means for providing selective access to said database 
elements using complexity level concept filtering; 

means for providing an interactive phrase memoriza- 
tion process using an order parts activity; 

means for providing an interactive phrase memoriza- 
tion process using a progressive typing activity; and, 

means for monitoring and reporting student progress 
and study history using individualized automated 
student monitoring and reporting. 

2. The technology assisted learning system as in claim 1 
wherein said computerized database elements comprise lan- 
guage elements. 

3. A method of technology assisted learning comprising 
the following steps: 

obtaining at least one computer comprising a visual 
display device, a central processing unit (CPU), one or 
more input devices, one or more output devices, and 
one or more memory portions; 

said memory portion(s) having software which performs 
the steps of: 

building and maintaining a computerized database 
comprised of database elements to be learned and 
their characteristics; 

conducting learning aaivities that will access said 
database for learning content; 

tagging said database elements to identify them as 
containing specified conceptual principles compris- 
ing a concept tagging system; 

presenting said elements using systematically spaced 
review and testing in a pool leaming system; 

providing selective access to said leaming activities 
using student specific activity sequencing; 
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providing selective access to said database elements 

using complexity level concept filtering; 
providing an interactive phrase memorization process 
using an order parts activity; 
s providing an interactive phrase memorization process 
using a progressive typing activity; and, 
monitoring and reporting student progress and study 
history using individualized automated student 
monitoring and reporting. 
10 4. The technology assisted learning system as in claim 3 
wherein said computerized database elements comprise lan- 
guage elements. 

5. A technology assisted learning system stored via stor- 
age media, the storage media comprising: 

15 a first plurality of binary values for building and main- 
taining a computerized database comprised of database 
elements to be learned and their characteristics; 
a second plurality of binary values for conducting leam- 
ing activities that will access said database for learning 
content; 

a third plurality of binary values for taggmg said database 
elements to identify them as containing specified con- 
ceptual principles comprising a concept tagging sys- 

25 ^^"^i 

a fourth plurality of binary values for presenting said 
elements using systematically spaced review and test- 
ing in a pool learning system; 

a fifth plurality of binary values for providing selective 
30 access to said learning activities using student specific 
activity sequencing; 

a sixth plurality of binary values for providing selective 
access to said database elements using complexity level 
concept filtering; 

a seventh plurality of binary values for providing an 
interactive phrase memorization process using an order 
parts activity; 

an eighth plurahty of binary values for providing an 
interactive phrase memorization process using a pro- 
gressive typing activity; and, 

a ninth plurality of binary values for monitoring and 
reporting student progress and study history using 
individualized automated student monitoring and 
45 reporting. 

6. The technology assisted learning system as in claim 5 
wherein said computerized database elements comprise lan- 
guage elements. 

7. A technology assisted language learning system com- 
50 prising: 

at least one computer comprising a visual display device, 
a central processing unit (CPU), one or more input 
devices, one or more output devices, and one or more 
memory portions; 
55 said memory portion(s) comprising: 

means for building and maintaining a computerized 
database comprised of language elements to be 
learned and their characteristics; 
means for conducting learning activities that will 
60 access said database for learning content; and 

means for presenting said elements using systemati- 
cally spaced review and testing in a pool learning 
system. 

8. The technology assisted language learning system of 
65 claim 7 further comprising means for tagging said database 

elements to identify them as containing specified conceptual 
principles comprising a concept tagging system. 
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9. The technology assisted language leaming system of 
claim 7 further comprising means for providing selective 
access to said learning activities using student specific 
activity sequencing. 

10. The technology assisted language learning system of 5 
claim 7 further comprising means for providing selective 
access to said database elements using complexity level 
concept filtering. 

11. The technology assisted language learning system of 
claim 7 further comprising means for providing an interac- 
tive phrase memorization process using an order parts 
activity. 

12. The technology assisted language learning system of 
claim 7 further comprising means for providing an interac- 
tive phrase memorization process using a progressive typing 
activity. ^5 

13. The technology assisted language learning system of 
claim 7 further comprising means for monitoring and report- 
ing student progress and study history using individualized 
automated student monitoring and reporting. 

14. A method of technology assisted language learning 
comprising the following steps: 

obtaining at least one computer comprising a visual 
display device, a central processing unit (CPU), one or 
more input devices, one or more output devices, and 
one or more memory portions; 

said memory portion(s) having software which performs 
the steps of: 

building and maintaining a computerized database 

comprised of language elements to be learned and 

their characteristics; 
conducting learning activities that will access said 

database for learning content; and 
presenting said elements using systematically spaced 

review and testing in a pool learning system. 

15. A technology assisted language learning system stored 
via storage media, the storage media comprising: 

a first plurality of binary values for building and main- 
taining a computerized database comprised of language 
elements to be learned and their characteristics; 

40 

a second plurality of binary values for conducting learn- 
ing activities that will access said database for learning 
content; and 

a fourth plurality of binary values for presenting said 
elements using systematically spaced review and test- ^5 
ing in a pool learning system. 

16. A technology assisted language learning system com- 
prising: 

at least one computer comprising a visual display device, 
a central processing unit (CPU), one or more input 50 
devices, one or more output devices, and one or more 
memory portions; 
said memory portion(s) comprising: 

means for building and maintaining a computerized 

database comprised of language elements to be 55 

learned and their characteristics; 
means for conducting learning activities that will 

access said database for learning content; and 
means for tagging said database elements to identify 

them as containing specified conceptual principles 60 

comprising a concept tagging system. 

17. A technology assisted language learning system com- 
prising: 

at least one computer comprising a visual display device, 
a central processing unit (CPU), one or more input 65 
devices, one or more output devices, and one or more 
memory portions; 
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said memory portion(s) comprising: 

means for building and maintaining a computerized 
database comprised of language elements to be 
learned and their characteristics; 
means for conducting learning activities that will 
access said database for learning content; and 
means for providing an interactive phrase memorization 
process using a progressive typing activity. 

18. A technology assisted language learning system com- 
prising: 

at least one computer comprising a visual display device, 
a central processing unit (CPU), one or more input 
devices, one or more output devices, and one or more 
memory portions; 
said memory portion(s) comprising: 

means for building and maintaining a computerized 
database comprised of language elements to be 
learned and their characteristics; 
means for conducting learning activities that will 

access said database for learning content; and 
means for providing an interactive phrase memoriza- 
tion process using an order parts activity. 

19. A technology assisted learning system comprising: 
at least one computer comprising a visual display device, 

a central processing unit (CPU), one or more input 

devices, one or more output devices, and one or more 

memory portions; 
said memory portions comprising: 

software for prompting for and receiving a response 
from a student, the response being directed to learn- 
ing one or more items; 

software for evaluating a student's level of comprehen- 
sion of said items; 

software for sorting said items based on said student's 
level of comprehension. 

20. The technology assisted learning system of claim 19 
wherein said prompting and receiving are accomplished 
through one or more learning activities. 

21. The technology assisted learning system of claim 19 
wherein said evaluating is performed using a real-time 
response evaluation. 

22. The technology assisted learning system of claim 19 
wherein said sorting is accomplished through a pool learning 
system. 

23. The technology assisted learning system of claim 19 
further comprising software for re-presenting said items to 
said student for review and re-evaluation using a systemati- 
cally spaced review. 

24. A technology assisted learning system comprising: 
at least one computer comprised of a display portion, a 

central processing unit (CPU) portion, an input portion, 
an output portion, and one or more memory portions; 
said memory portion(s) comprising: 

(a) software for presenting a plurality of items to be 
learned; 

(b) software for evaluating a student's level of com- 
prehension of said items; 

(c) software for sorting said items into a plurality of 
pools based on said student's level of comprehen- 
sion; 

(d) software for reviewing said items, from each of said 
pools, at a frequency and a level of difficulty deter- 
mined by said student's level of comprehension 
associated with each of said pools, said frequency 
being higher for a lower level of comprehension and 
lower for a higher level of comprehension and said 



03/16/2004, EAST Version: 1.4.1 



6,077, 

31 

level of difficulty being higher for a higher level of 
comprehension and lower for a lower level of com- 
prehension. 

(e) software for evaluating said student's new level of 
comprehension during said review; 5 

(f) software for re-sorting said concepts into said pools 
based on said new level of comprehension; and 

(g) software for repeating steps (d), (e) and (f) until a 
desired level of comprehension is attained. 

25. The Technology assisted learning system of claim 24 
wherein said software for presenting a plurality of items to 
be learned displays examples of the item to be learned, 
provides a definition or explanation of the item to be learned, 
and provides activities wherein the student can develop an 
understanding of the item. 

26. The technology assisted learning system of claim 24 
wherein said evaluating said student's level of comprehen- 
sion and said smdent*s new level of comprehension are 
determined by said student's speed of response and correct- 
ness of response. 

27. The technology assisted learning system of claim 24 20 
wherein said evaluating said student's level of comprehen- 
sion and said student's new level of comprehension are 
determined by said student's quantity of consecutive correct 
responses, quantity of correct responses in a given session, 
speed of first response to a question when answered 25 
correctly, and percent of correct responses in a current 
review session. 

28. A data indexing system comprising: 

at least one computer comprised of a display portion, a 
central processing unit (CPU) portion, an input portion, 3Q 
an output portion, and one or more memory portions; 
said memory portion(s) comprising: 

software for assembling a database composed of data- 
base elements; 
software for identifying one or more learning concepts; 35 
software for analyzing each of said database elements 
to determine whether it contains any of said learning 
concepts; and, 
software for tagging each of said database elements for 
each of said learning concepts contained therein so 40 
that database elements can be identified and accessed 
by the concepts contained therein. 

29. The system of claim 28 wherein said database ele- 
ments are composed of words, phrases, sentences, or other 
language constructs, 45 

30. The system of claim 28 wherein said database ele- 
ments are composed of mathematical expressions. 

31. The system of claim 28 wherein said learning con- 
cepts are grammar principles. 

32. The system of claim 28 wherein said learning con- 50 
cepts are mathematical principles. 

33. The system of claim 28 wherein said learning con- 
cepts are vocabulary words. 

34. An information learning system comprising: 

at least one computer comprised of a display portion, a 55 
central processing unit (CPU) portion, an input portion, 
an output portion, and one or more memory portions; 
said memory portion(s) comprising: 

software for dividing a block of information into indi- 
vidually meaningful parts; 60 
software for removing from said individually meaning- 
ful parts designated punctuation, capitalization or 
other superfluous matter that would help disclose the 
original sequence of said block of information; 
software for presenting to a user said individually 65 
meaningful parts in a sequence different from the 
original sequence of said block of information; 
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software for prompting the user to assemble said indi- 
vidually meaningful parts into said original sequence 
of said block of information; 

software for determining if the student assembled said 
individually meaningftil parts into said original 
sequence of said block of information; 

software for further subdividing said individually 
meaningful parts if the student successfully 
assembles said parts into said original sequence and 
rcprompting the user to assemble the parts into said 
original sequence of said block of information; and 

software for partially combining said individually 
meaningful parts into said original sequence of said 
block of information if the user fails to assemble said 
individually meaningful parts into said original 
sequence and reprompting the user to assemble the 
parts into said original sequence of said block of 
information. 

35. The information learning system of claim 34 wherein 
said block of information is a phrase or sentence in a 
language to be learned, and said prompting the user to 
assemble said individually meaningful parts into said origi- 
nal sequence is a native language translation of the block of 
information. 

36. The information learning system of claim 34 wherein 
said sequence different from said original sequence of said 
block of information is an alphabetical sequence of said 
individually meaningful parts. 

37. The information learning system of claim 34 wherein 
said software for further subdividing said individually mean- 
ingful parts adds distractor parts when said individually 
meaningful parts are divided into basic information elements 
that cannot be further subdivided while retaining meaning. 

38. An information learning system comprising: 

at least one computer comprised of a display portion, a 

central processing unit (CPU) portion, an input portion, 

an output portion, and one or more memory portions; 
said memory portion(s) comprising; 

software for prompting the student for an open-ended 
response to a question; 

software for analyzing said response for correctness; 

software for displaying the correct response wherein 
incorrect portions of the user's response are dis- 
played as dashed lines with one or more of the 
correct characters displayed as hints and wherein 
correct portions of the user's response are displayed 
in their correct form; 

software for permitting the student to make a subse- 
quent response to provide the complete hinted word 
or words; and 

software for displaying the correct response wherein 
incorrect portions of the user's response are dis- 
played as dashed lines with one or more of the 
correct characters displayed as hints and wherein 
correct portions of the user's response are displayed 
in their correct form, repeating the response and 
display routine until the user is able to produce the 
phrase without any hint characters showing. 

39. The information learning system of claim 38 wherein 
said question is a phrase or sentence in a language to be 
learned, and said prompting the user to provide the open- 
ended response is a language translation. 

40. The information learning system of claim 38 wherein 
said one or more of the correct characters displayed as hints 
comprise the characters at the beginning of the conrect 
response. 

41. The information learning system of claim 38 wherein 
said one or more of the correct characters displayed as hints 
comprise all but one of the characters of the correct 
response. 
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42. An information learning system comprising; 
at least one computer comprised of a display portion, a 
central processing unit (CPU) portion, an input portion, 
an output portion, and one or more memory portions; 
said memory portion(s) comprising: ^ 
software for creating and maintaining a database com- 
prised of database elements which are linked to 
conceptual content examples; 
software for allowing input of a query of one or more 

parts representing a desired concept; 
software for searching said database for elements that 
represent each part of a given query and recording 
said elements in element lists corresponding to each 
query part; 

software for deleting from said element lists the ele- 
ments that are not linked to a content example that is 
linked to other elements representing all other parts 
of the query; 

software for displaying said element lists to a user and 
for rejection of elements by allowing said user to 
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manually reject elements which do not correspond to 
the desired concept; 
software for automatically eliminating from said cle- 
ment lists elements linked to a content example that 
is no longer linked to other elements representing all 
other parts of the query due to said rejection of 
elements; 

software for finding all remaining content examples 
that are linked to one remaining element in each 
clement list; 

software for displaying said remaining content 
examples to the user and for manual rejection and 
creation of a final content example group by allow- 
ing the user to reject said remaining content 
examples which do not correspond to the desired 
concept; and, 

software for storing said final content example group 
for subsequent access according to the concept it 
represents. 

* * * )|t 4 
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DATED : June 20, 2000 

INVENTOR(S) : Kent Parry and C. Bret Elzinga 



It is certified that error appears In the above- Identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Title page. 

Item [56], References Cited, change "5,957,669" to -- 5,957,699 

Item [57], ABSTRACT, 

Line 26, change "students" to — student's - 

Drawings, 

Figure 9, Reference 960, change "their" to -- the both instances. 
Column 1, 

Line 24, change "her" to -- his/her - 
Line 56, change "her" to -- his/her -- 
Line 60, change "her" to -- his/her -- 

Column 2. 

Line 16, change "allows" to allow -- 
Line 21, change "allows" to -- would allow -- 

Column 3, 

Line 3, change "her" to -- his/her - 
Column 11, 

Line 18, change "or" to - of — 
Column 12, 

Line 22, change "of" to -- or -- 
Line 26, change "her" to - his/her -- 

Column 25, 

Line 47, change "as" to - a - 
Column 26, 

Line 20, change "her" to - his/her -- 
Line 21, change "she" to -- he/she -- 
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